This document uses the terms ‘content,’ media′ and ‘media content’ interchangeably to refer to pictures, music, movies and all of their sundry creative brethren which collectively might fall under the grand umbrella named ‘creative works’. In general, a given creative work can be thought of as an entity or ‘content object’, and either the creative work stands alone with the sole companion of a name or an identification number, or it is somehow duplicated and packaged for a very wide variety of distribution methods and channels.
In primarily the latter case, the concept of ‘metadata’ has gained popularity, referring to additional creative works which have some form of explicit or implicit relationship to a given singular content object. Metadata generally refers to information associated with a content object. Typically, metadata is often associated with multimedia content, like images, and video and audio programs, and is used to refer to information about the content, such as its source, owner, content title, etc. One of the simpler forms of metadata might be bit-fields describing additional information about a content object such as an author's name or a category that the content object might naturally fall under. More complicated forms of metadata might be ‘pointers’ or addresses (e.g. URLs) of related content objects which, by reference, enable a user or consumer to easily access that second content object. Most generally, a key concept from a content provider and/or content distributor's point of view is that metadata can form an instant electronic relationship between a consumer experiencing one of their distributed content objects and themselves.
In this document, metadata refers to a broad class of information relating to a content object, and it applies to a broad class of content objects, including both physical and electronic objects. Metadata also includes an instruction or set of instructions (possibly distributed over one or more devices) that is executed by machine or machines to perform a behavior associated with an object (e.g., perform an on-line transaction, transmit or transfer content, authenticate/verify a user, content, access token, update/patch a program etc.). Metadata can be formatted and stored in a variety of formats. One format is XML, but there are others. The metadata for a particular content object may be distributed over different storage devices. In such a distributed storage approach, the metadata in one location includes references to metadata in other locations (such an index, pointer, address, URL, etc.).
One aspect of the invention relates to the technical infrastructure, including metadata routing and associated network services that ensure this electronic relationship can happen in the first place, and that this relationship can lead toward secondary revenue generation opportunities that may some day rival and eclipse primary media distribution revenue generation.
The business of selling packaged media or otherwise delivering specific media to a targeted audience has a long history of monetizing the primary delivery of that media. Selling records or selling 30 seconds of advertising on a television show or selling tickets to a movie each fit into the primary distribution monetization business model. The growth in the Internet and the flowering of various digital distribution channels certainly has complicated the description of primary media distribution itself, but the general notion of “packaging up” a creative work and delivering it for some explicit compensation strategy remains intact.
In a kind of direct extrapolation of the primary distribution model, Digital Rights Management (DRM) inventions and approaches have at least a decade worth of effort, design, trial, partial successes and valuable lessons now under their collective belt. Not least in these lessons are the behind-the-scenes business community wrestling and clashes of Titans surrounding the question on what might be considered a core property of DRM approaches: “who owns the standard . . . who owns the channel . . . who ultimately owns the consumer relationship?” Phrased in this way, there can be little mystery why uniform global standards are possibly decades away still.
Two somewhat different forces have arisen in the past decade or two which have not been entirely harmful to classic primary media distribution monetization, but they have nevertheless put significant pressure on businesses which rely on primary distribution revenue to look for secondary methods of monetization. At the very least, these forces have led toward fundamental changes of strategy in how primary distribution methods are exploited.
The first force is the ease with which creative works can be copied and re-distributed in an unauthorized fashion. The second force is the advent of highly distributed media distribution channels and the equally highly distributed end-devices used to experience a creative work, most certainly including mobile devices. Though these two forces are rather different from each other and though each has been partially transformed into “opportunity” by ever-entrepreneurial efforts and companies, the fact remains that both forces are unstoppably disrupting traditional approaches to the monetization of primary media distribution.
The relative ease with which creative works can be copied has been a primary fuel in creating the now familiar notion of peer-to-peer networks where folks not only share pictures from family vacations but also the latest movie they enjoyed last night. Untold years of technological ponderings and industry standards initiatives have sought to re-establish the core role of primary packaged media distribution and its associated monetization, also not without some success, but the genie does seem to be rather out of the bottle for those seeking to re-create the good old days of creation-to-consumption monetization. Coordinated primary and second monetization strategies and cash-generating mechanisms are inevitably here to stay, most likely co-opting the second force of highly distributed distribution channels and consumption devices. Of particular note is the up and coming ‘mobile’ media consumption trend where ubiquitous connectivity meets ubiquitous delivery.
Still missing in this inevitable balancing of primary and secondary monetization methods are the critical details of the secondary monetization methods and systems, as well as the impact of their existence on primary distribution methods and strategies. In other words, how can secondary monetization work (beyond peddling primary distribution of ring tones), and how can primary distribution production processes be seamlessly modified to put the overall media industry profit and revenue lines back onto positive and strongly growing paths? This document describes a routing system and method, along with detailed hardware and software descriptions of the system and network components which can make it work in a tremendously complicated media distribution and consumption universe.
Managing the Fountain of Metadata
Once a distributor has accomplished primary delivery of a media object to a consumer, the next best thing to “manage” is the metadata services that add value to that delivered object. This is not the slippery slope named “control” which is a word written on many a tombstone of the last decade's worth of packaging and monolithic DRM-system approaches to media consumption behaviors, it is fundamentally about managing the highest quality relationship to the individuals or groups who are naturally attracted to the media content in the first place. The initial packaging up of “good stuff” metadata into the primary delivery of the original media content was one of the main reasons a consumer will pay a modest amount for the officially sanctioned media, but it will be the ongoing access to high quality metadata, group affinities and seamless access to related media which will compel honest consumers to be honest . . . they will simply get more value for their time and money that way.
Clearly, combining a media object with static metadata or even “static links” to inherently dynamic web-based content is a developed art at this point and somewhat accounts for the ongoing vitality of primary distribution channels over some P2P network channels. In other words, packing in “good extra stuff” still sells records and movies and pictures. P2P copies can try to keep up, but the rightful distributor has a leg up on the availability of legitimately compelling content, precisely because they are the legitimate owners or distribution rights-holder.
In some advertising-based business models, content objects are distributed for free or reduced cost and provide a vehicle for advertising revenue. These models provide an opportunity for content owners to monetize content by conveying advertising within the content. Yet, to capitalize on this opportunity in distribution networks like wireless networks and the Internet, there is a need for mechanisms to tie the content consumption to revenue opportunities, such as linking the content to electronic transactions to buy related content or products and services advertised within the content.
Fortunately, there are a few common denominators of all media distribution and media consumption that will never go away and which get to the heart of ensuring a stable relationship between content providers and content consumers. One common denominator is simply identity of the content itself. Another common denominator is the existence of basic business rules and legal frameworks which collectively define the common sense notion of “legitimate distribution and consumption” and its associated notion of return on investment to the content providers. A third common denominator is the near-universal desire of consumers to have access to the best information related to the content being consumed. And finally, there is a fourth common denominator that a content provider wants to own the “rules of relationship” associated with the content they distribute: Business rules and contracts should define how legitimate and consumer-friendly metadata relationships are carried out.
One aspect of this disclosure refers to this notion as the title of this section indicates: managing the fountain of metadata that a consumer wants and will eventually expect. The other side of this coin is classic business principle behind satisfying customers: delivering the highest quality “rewards” in managing this fountain will ensure repeat business.
The raw mechanics of how this management can happen in the global cacophony of media flow provides the technical foundation for these emerging content distribution and monetization models. The careful reader will see that the described system and network mechanisms are fully complementary to DRM-based approaches in the “digitally contained” world of the Internet and classic dedicated media channel delivery networks on the one hand, and fully able to deal with complexities and steep growth of mobile device consumption on the other hand.
In one embodiment illustrated in FIG. 9, a routing system includes two primary processing engines that are used to link a media object held by a consumer to a source of metadata. The first engine, including the ID resolver and registry components, can be described as the lingua franca of identification systems, methods, technologies, etc. It can also be effectively described as a virtual “DNS for Content Objects,” as this identification engine respects any and all native or monolithic approaches to content identification. Hence the qualifier “virtual” in front of DNS. Its function is to resolve a content ID based on identifying information originating from disparate content identification systems.
The second engine, the rules database and processor, determines where to re-direct the consumer based on the resolved content ID. This rules engine facilitates secondary revenue generation opportunities because it further enables the system to tailor the metadata response to provide related content, products and services. The content provider universe is complex and often has a wide variety of business interests at play. Such interests are most often encapsulated in contracts between various entities, including the artists which create works in the first place, and such contracts can be extended to the detailed rules of metadata response to normal and/or pro-active metadata requests during media consumption sessions. The quality of the response to the consumer, and the opportunity to direct consumer's to additional revenue generating activities including classic eyeballs/advertising pathways, can be enabled by this rules engine.
FIG. 9 shows the first and second engines as being part of a router system, these engines can be partitioned and distributed over devices and controlled by different participants. The rules processor and database may be partitioned from the router system and implemented in separate instantiations, each controlled by a different participant. In this case, for example, the resolver re-directs a consumer to a rules engine under the control of a participant linked to the object via the ID registry, and this rules engine, in turn, executes a rule that determines the metadata response for the consumer (e.g., a URL or set of URLs to particular metadata). The rules processor may also be executed, at least in part on a device under the client's control. In this case, for example, a set of URLs linked to the content object via the ID registry are returned to the client, which in turn, executes rules to determine the metadata response tailored to the consumer.
The good old days of selling packaged media is still with us. The content being sold is now the seed for an ongoing relationship in ways that classic “branding” could barely fathom. This disclosure details how these two core engines can be built and operated for the good of content providers and content consumer's alike.
Managing the Relationship Between Metadata and Content Objects
As noted, metadata plays an important role in managing and facilitating transactions in content objects. Some significant examples include the use of metadata in digital distribution of content, electronic commerce, and on-line searching and organization of vast stores of data (e.g., the Internet). As the digital world proliferates and there are numerous transactions in content objects, there is a compelling need to manage the association of metadata and content objects.
This need is not confined to the digital realm. Because humans live in the physical and analog realm, there will always be a need for efficient schemes for crossing back and forth between the digital and analog realms. In particular, physical objects have corresponding metadata just as electronic objects do. For example, products have corresponding metadata in the form of product information, manuals, catalogs of related products, etc. Printed objects have metadata in the form of electronic versions of the object, ownership, source, time and location of creation, etc. Physical objects link to their metadata via an identifier on or derived from the product or related documentation (e.g., packaging, labels, etc.). Metadata management technologies, thus, need to be able to support this physical/electronic interface. Emerging applications include linking physical objects to Internet related information and electronic transactions as described in U.S. Pat. Nos. 6,947,571 and 6,505,160 and International Patent Application WO 97/43736, which are incorporated by reference.
A significant aspect of managing metadata of disparate content objects is providing effective technologies and schemes for content identification. This is important in the digital realm, where there are many potentially conflicting content identification technologies and architectures. It is also important for managing metadata for physical objects in the digital realm, where identifiers extracted or derived from physical objects provide a form of digital identity of the physical object in the digital world. The metadata systems and methods in this document are designed to work with identification systems that operate in the digital realm only, as well as ones that span the digital and physical realms. The latter category includes identification methods that derive content identifiers from an electromagnetic signal captured from an analog representation of audio or images (e.g., a digital watermark, content fingerprint, visual symbology, pattern recognition, voice recognition, OCR, etc.) as well content identifiers read via electromagnetic readers of physical data carrier devices like magnetic stripes (and other magnetic data carriers), RF ID tags, smart cards, etc. For example, physical object can be identified via RFID tags, as described at www.epcglobaline.org and in the overview document (www.epcglobaline.org/news/EPCglobal_Network_Overview_10072004.pdf.), which is incorporated by reference.
Such content identification technology provides a means to identify content objects, but the variety of content identification schemes and formats poses compatibility and interoperability challenges. Moreover, such systems cannot provide useful information without an effective system and method to associate various identifiers with the appropriate metadata.
The problems are multifold and created by the fact that digital distribution separates content from packaging, new 1-1 marketing opportunities are minimally being utilized, and digital distribution is moving forward with proprietary channels that make the value chain more complex rather than simpler.
For instance, once content is digitized information typically carried on physical packaging is lost from the content. Digital downloads are partial products, “files without packaging and related metadata”. Metadata loss is central to issues surrounding digital content management, piracy and e-commerce. Manual population of multiple distribution channels' metadata repositories gives rise to human error and inaccurate metadata.
Marketing opportunities are being lost once content is distributed since content owners and retailers lose contact with the consumer. Loss of 1-1 marketing capabilities, especially with digital distribution gaining traction, leads to loss of potential revenue.
Channels of distribution (e.g., online music retailers, podcasts, social networking sites, user-generated content sites, and P2P networks) and the number of digital derivatives (ring tones, mobile videos, etc.) stemming from a single digital product are increasing. Accurate and effective content identification is an absolute requirement to manage content effectively. Content owners are currently evaluating their metadata repositories trying to understand how to streamline in a manner that is cost-effective. Proprietary content identification and metadata systems complicate, rather than simplify, the value chain. Content is embedded with many identifiers that do not interoperate. A few proprietary systems are linking content to metadata without input of the content owners, thus increasing the number of value chain participants.
Previous initiatives to create a central content metadata repository have failed due to proprietary, political and technical issues of creating a repository rather than directory service. Content owners and retailers want to manage their proprietary metadata and participate in building the relationship with the consumer. Third party metadata companies, and related companies, such as those that organize, classify, search and provide search results based on metadata (such as search engine providers), stand to profit from potential unauthorized use of content owners' metadata.
This document describes systems and methods for associating metadata with content objects. It describes embodiments of novel routing methods and systems referred to as content metadata directory services.
Globally Unique Identifier Scheme
One novel method of associating a content object with metadata uses a combination of a content identifier and a bounding identifier to enable handling of disparate sets of content identifiers for content objects with potentially conflicting content identifiers. The method receives a content identifier for a content object from among a set of content identifiers. It provides a unique bounding identifier for the set of content identifiers. This unique bounding identifier is used in combination with the content identifier to form a globally unique identifier for the content object. This globally unique identifier is associated with a metadata source, which enables routing of a user to the metadata source.
This approach effectively manages cases where an ID provider pre-assigns a set of content identifiers to objects, and then later registers them in our novel directory system. It also manages cases where the directory system assigns the content identifier prior to insertion of the content identifier in the content object by an ID provider.
As set forth in the CMDS embodiments, the unique bounding identifier may comprise an ID provider identifier. For example, RFID, EPC, digital watermarking and fingerprinting technology providers can serve as ID providers in the system with overlapping content ID numbers, but unique ID provider IDs. Each ID provider may also use an ID version to distinguish different versions of its technology or content ID spaces.
After appropriate registration, the directory system is used to route users to a metadata source. For example, the user (e.g., the reader executing on the user's device) provides the content ID from the content object and the bounding identifier. The directory system, in turn, routes the user to the metadata source associated with the globally unique identifier for the content object.
Metadata Directory Supporting Content Objects with Multiple Content Identifiers
Another novel method addresses content objects with two or more content identifiers, potentially referencing different metadata sources. This method registers different globally unique identifiers for a content object. These globally unique identifiers each comprise a content identifier provided with the content object and a bounding identifier identifying a set of content identifiers of which the content identifier is a member. For each of the globally unique identifiers, information is maintained about a metadata source. The method receives a first content identifier for the content object, and uses a bounding identifier associated with the set of the first content identifier to determine the globally unique identifier for the first content identifier. The user is routed to the metadata source associated with globally unique identifier.
This approach handles a variety cases in which two or more content identifiers are provided for a content object for the purpose of registration or resolution. The metadata directory system supports and manages both the registration of and routing to different metadata sources corresponding to different content identifiers of the content object. These cases include:
1. Content identifiers are embedded or calculated by different ID providers and are later derived from the content object using different readers associated with those technologies. For example, the readers are different because they derive the content identifier using different content identification methods (e.g., through the file header/footer, digital watermark, fingerprint, Vertical Blanking Interval data in video programming, etc).
2. The different readers may, for example, derive the content identifiers using different attributes of the content object. These different attributes may comprise different types of embedded auxiliary data (different watermark embedders/readers, watermark vs. embedded header/footer data). These different attributes may comprise attributes from which different digital watermarks or robust hashes are derived. The different attributes may correspond to in band and out of band attributes of the content object. “In band” refers to an identifier derived from content in the content object that is rendered for perception by a human. “Out of band” refers to auxiliary data carried in the content object but not forming part of the content that is rendered for perception by a human. Certain types of content objects include multiple content programs rendered for perception by a human, like video and audio tracks and close captioned text. In band identifiers may be derived from one or more of these content signals within the content object. In some cases, one content program may be embedded in another content program within a single content object, such as where close captioned text is embedded in the audio or video program of an audiovisual work.
3. The different content identifiers for a content object may be derived from the content object using different parts of the content object, including different in band and out of band parts as well as different parts within the in band portion of the object and different parts within the out of band portion. These parts may be in discrete locations in one domain of the content signal, yet at overlapping locations in others. Examples of domains include spatial, temporal and transform domains (e.g., frequency domain, compressed domain, etc.) of the content signal in a content object).
Enabling Different ID Provider and Content Provider Participants
In some metadata systems, the system owner, serving as a registration authority (RA), provides the identification technology and content owners use the technology to register themselves as a content provider, register content and link the content to metadata.
This document describes a novel system that enables multiple identity providers (ID Providers) to register and use the system. The ID Provider registers with a metadata directory system, receives a unique bounding identifier, and uses this bounding ID (e.g., an ID provider ID) with subsequent interactions with the metadata directory system. Separately, metadata source providers register metadata sources with the metadata directory system. This enables many different participants to associate content objects with metadata sources using one or more identity providers. Examples of metadata source providers include content providers, like content owners or retailers that have the flexibility of working with different ID providers to associate content objects with metadata. Both content providers and ID providers can register and use the system. The metadata source is the system or device that provides the metadata, like a web site. The directory system uses an identifier for the metadata source, which enables it to maintain an association between a content object and its corresponding metadata source. For example, in some embodiments, a URL serves to identify the location of the source.
One embodiment of the directory system is referred to as CMDS. CMDS enables content providers to utilize the CMDS to knit together metadata sources that are associated with content using disparate and previously incompatible ID provider technologies. CMDS enables content providers to manage their proprietary information (i.e. they do not have to turn over control of proprietary metadata to a RA for storing and distributing the metadata), enables eCommerce for all value chain participants (e.g., both content owners and retailers can embed CIDs), facilitates interoperability with all content identity provider technology (even pre-existing ID systems, such as EPC), allows for compatibility with both PC and mobile devices, facilitates interoperability for multiple ID providers who license a common identification algorithm, and enables usage reporting and vital marketing statistics.
One aspect of the invention is a system for processing audio to provide metadata responses to different mobile device application programs with corresponding content ID schema. Audio signals are processed in a distributed reader and metadata routing system to provide metadata responses to mobile application programs. A routing system registers identifiers for different content ID schema. These identifiers are encoded in audio signals. Mobile device application programs are equipped with reader programs to extract identifiers from audio sensed by the microphone of a mobile device. Metadata responses are identified by the ID provider of the content ID schema and extracted identifiers. The metadata routing system routes metadata responses to a requesting mobile device, supporting a variety of different ID providers and mobile device applications.